A Model-based Evaluation of Trust and Situation Awareness in the Diner’s Dilemma Game

This paper describes a machine learning approach to evaluate the relationship between trust behavior and Situation Awareness (SA) in the context of a 3-player Iterated Diner’s Dilemma game. Our experimental setup consisted of a set of 24 supervised studies in which participants played against computer opponents with different cooperation strategies. Three user interfaces were evaluated in the study, each corresponding to a specific level of SA (Perception, Comprehension, and Projection). Two concepts of trust are explored in the study: Empirical trust behavior was recorded as the degree of cooperation imparted by the participant in a given condition, and Self-reported trust assessments were also collected at regular intervals throughout the study. To explore the relations that exist between these two concepts of trust, and different SA Level conditions, a machine learning approach was applied to train a variety of models to accurately predict trust behavior in each condition. Our best performing algorithm was a J48 rule-based learner, which leveraged SA Level, strategy, and self-reported survey data to predict trust behavior to 76% accuracy, and 67% accuracy using only SA Level and opponent strategy. This is a relative increase of 43% and 24% respectively over a benchmark majority class predictor. These results indicate that support at a level of SA and opponent strategy are good predictors of trust behavior in the Iterated Diner’s Dilemma game. Our results also show that trust-prediction models perform best when they are trained on a combination of self-reported data, and strategy / SA Level information.